Architecture For Semantic Segmentation Research Articles

Spectral–Spatial transformer-based semantic segmentation for large-scale mapping of individual date palm trees using very high-resolution satellite data

Date palm plantations in the United Arab Emirates (UAE) are under threat from soil salinity, drought, and date palm weevils. Accordingly, monitoring and conserving date palms are crucial to preserving a vital component of the country’s agricultural heritage, economy, food security, and ecological balance. Previous studies have effectively identified date palm trees using RGB-based aerial and UAV imagery utilizing diverse deep learning methods. However, the utilization of very high-resolution satellite data for delineating individual date palm crowns remains unexplored due to the limited spatial resolution capabilities of existing satellite systems. This study primarily aimed to achieve precise and comprehensive mapping of date palm trees using WorldView-3 (WV-3) satellite data by leveraging the high representational power of the state-of-the-art vision transformers (ViT) in capturing global information from the input data. First, an in-depth analysis assessment of the various transformer-based semantic segmentation architectures, including UperNet with vision transformer and Swin transformer, SegFormer, Mask2Former, and UniFormer, was conducted. Second, the integration of spectral data on the performance of ViTs was evaluated. Moreover, the models’ generalizability and complexity effect on the segmentation effectiveness were assessed. Accordingly, a postprocessing strategy was developed to aid in delineating and counting date palm trees from semantic segmentation outputs. Results demonstrated that integration of WV-3 spectral data into the analysis resulted in a marked improvement in segmentation quality. The UniFormer, UperNet-Swin, and Mask2Former models demonstrated considerable improvements in multispectral data analysis, with increases in mean intersection over union (mIoU) of 2.17% (77.88% mIoU, 86.01% mean F-score [mF-score]), 2% (78.10% mIoU, 86.18% mF-score), and 1.15% (77.36% mIoU, 85.59% mF-score), respectively, compared with their RGB-based results. Evaluations of model transferability also indicated that Mask2Former, UniFormer, and UperNet-Swin transformers efficiently adapted to multispectral data in the Dibba region. These models achieved mIoU scores of 84.36%, 84.25%, and 83.17% and mF-scores of 90.95%, 90.87%, and 90.13%, highlighting their effectiveness and potential for broader regional application. This research highlights the efficacy and feasibility of using ViTs with WV-3 multispectral data for accurate and comprehensive surveying of date palm plantations, enabling the development of palm tree inventories and continuously updating geospatial databases.

WasteNet: A novel multi-scale attention-based U-Net architecture for waste detection in UAV images

Waste littering contributes to various ecological, aesthetic, economic, and social hazards. Given the dispersed nature of littered waste in public spaces, there is a need for a wider area of inspection to enable efficient detection of such waste. The study re-emphasizes the utility of UAVs for round-the-clock public space monitoring. The application of UAVs provides constant vigil over large public spaces. However, detecting the UAV-captured objects with precision is challenging as the far-off view further reduces the size of the target waste. Moreover, the waste objects have a multi-scale nature with different shapes, textures, and compositions. Given these varied challenges, this paper proposes a novel multi-scale attention-based deep network architecture (WasteNet) for efficient semantic segmentation of UAV-collected images to detect waste objects. Attention-based multi-scale fusion blocks (ABFBs) are introduced to improve semantic segmentation performance. These ABFBs can learn the contribution of different convolution scales during the training phase. The performance of the proposed WasteNet has been validated against seven other existing state-of-the-art waste localization methods. Moreover, an ablation study has been performed to derive the best architecture. The experimental results witnessed that WasteNet can detect litter from UAV-based images with a precision of 0.99, recall of 0.99, and MAP@50 of 0.95. For a better understanding of the results, the visuals reflecting an improved precision of waste boundary identification have also been included.

Detection of karst depression in Brazil comparing different semantic and instance segmentations and global digital elevation models

This research assessed the efficacy of deep segmentation in identifying and measuring natural karst depressions in the Bambuí Group's carbonate rocks in Western Bahia, Brazil. The investigation used five Global Digital Elevation Models (DEMs) with 30-m resolution: Advanced Spaceborne Thermal Emission and Reflection Radiometer - Global Digital Elevation Model version 3 (ASTER-GDEM v3), Advanced Land Observing Satellite World 3D – 30 m version 3.2 (AW3D30 v3.2), Copernicus 30 m global DEM (GLO-30), NASA Digital Elevation Model version 1 (NASADEM v1), and Shuttle Radar Topography Mission version 3 (SRTM v3). The karst feature detection analysis compared five semantic segmentation architectures: Feature Pyramid Networks (FPN), LinkNet, UNet, UNet++, DVL3+, using an EfficientNet-B7 backbone, one instance segmentation model (Mask-RCNN), and a semantic-to-instance conversion method. This comparison considers different datasets, including one, two, or eleven variables (DEM, DEM-based sink depth, and nine terrain attributes). Besides, we performed a combinatorial analysis of the DEMs to evaluate the improvement in karst feature detection. The methodology involved the generation of geomorphometric attributes, sample labeling from Sentinel-2 and Operational Land Imager-Landsat 8 datasets, training-validation-testing steps with 128 × 128 samples, reconstruction of large images for semantic and instance segmentation, and accuracy analysis. The findings revealed that GLO-30 and AW3D30 data were the most accurate, while ASTER GDEM performed poorly in both segmentation forms. Among the semantic models, FPN showed the highest accuracy. The 11-variable models preferentially outperformed those with fewer in both types of segmentation. The approach of semantic-to-instance conversion with Geographic Information System tools favored individualizing karst depressions and obtaining efficient quantification, consisting of an easy alternative to achieve instance segmentation. Reconstruction of large remote sensing images through sliding windows considering the specifics of instance and semantic segmentation demonstrated that smaller stride enhances object coverage and reduces the risk of losing crucial information, effectively improving the predictions. The combinatorial analysis for semantic and instance segmentation indicates that models incorporating more DEMs (4- and 5-DEM models) and variables achieve generally higher accuracy. The best result in semantic segmentation was combining the five DEM datasets using 11 variables, reaching an F-score of 85.06 and IoU of 74.00. In instance segmentation, the best result for the bounding box was the model that integrates AW3D30, GLO-30, NASADEM, and SRTM with eleven variables reaching Average Precision (AP) of 45.64, AP50 of 88.40 and AP75 of 39.38, while the best result for the segmentation mask was the model that integrates the five models with eleven variables with AP of 42.61, AP50 of 86.79 and AP75 of 35.43. These results demonstrate that integrating a wide range of DEM data, even the lowest performing ones, improves model generalization and accuracy in segmentation, leveraging strengths and mitigating individual weaknesses. Future work could explore high-resolution DEMs and the integration of various deep-learning methods.

Lunar ground segmentation using a modified U-net neural network

Semantic segmentation plays a significant role in unstructured and planetary scene understanding, offering to a robotic system or a planetary rover valuable knowledge about its surroundings. Several studies investigate rover-based scene recognition planetary-like environments but there is a lack of a semantic segmentation architecture, focused on computing systems with low resources and tested on the lunar surface. In this study, a lightweight encoder-decoder neural network (NN) architecture is proposed for rover-based ground segmentation on the lunar surface. The proposed architecture is composed by a modified MobilenetV2 as encoder and a lightweight U-net decoder while the training and evaluation process were conducted using a publicly available synthetic dataset with lunar landscape images. The proposed model provides robust segmentation results, allowing the lunar scene understanding focused on rocks and boulders. It achieves similar accuracy, compared with original U-net and U-net-based architectures which are 110–140 times larger than the proposed architecture. This study, aims to contribute in lunar landscape segmentation utilizing deep learning techniques, while it proves a great potential in autonomous lunar navigation ensuring a safer and smoother navigation on the moon. To the best of our knowledge, this is the first study which propose a lightweight semantic segmentation architecture for the lunar surface, aiming to reinforce the autonomous rover navigation.

Identification of optimal semantic segmentation architecture for the segmentation of hepatic structures from computed tomography images

Identification of optimal semantic segmentation architecture for the segmentation of hepatic structures from computed tomography images

Semantic segmentation using cross-stage feature reweighting and efficient self-attention

Recently, vision transformers have demonstrated strong performance in various computer vision tasks. The success of ViTs can be attribute to the ability of capturing long-range dependencies. However, transformer-based approaches often yield segmentation maps with incomplete object structures because of restricted cross-stage information propagation and lack of low-level details. To address these problems, we introduce a CNN-transformer semantic segmentation architecture which adopts a CNN backbone for multi-level feature extraction and a transformer encoder that focuses on global perception learning. Transformer embeddings of all stages are integrated to compute feature weights for dynamic cross-stage feature reweighting. As a result, high-level semantic context and low-level spatial details can be embedded into each stage to preserve multi-level information. An efficient attention-based feature fusion mechanism is developed to combine reweighted transformer embeddings with CNN features to generate segmentation maps with more complete object structure. Different from regular self-attention that has quadratic computational complexity, our efficient self-attention method achieves similar performance with linear complexity. Experimental results on ADE20K and Cityscapes datasets show that the proposed segmentation approach demonstrates superior performance against most state-of-the-art networks.

SasWOT: Real-Time Semantic Segmentation Architecture Search WithOut Training

In this paper, we present SasWOT, the first training-free Semantic segmentation Architecture Search (SAS) framework via an auto-discovery proxy. Semantic segmentation is widely used in many real-time applications. For fast inference and memory efficiency, Previous SAS seeks the optimal segmenter by differentiable or RL Search. However, the significant computational costs of these training-based SAS limit their practical usage. To improve the search efficiency, we explore the training-free route but empirically observe that the existing zero-cost proxies designed on the classification task are sub-optimal on the segmentation benchmark. To address this challenge, we develop a customized proxy search framework for SAS tasks to augment its predictive capabilities. Specifically, we design the proxy search space based on the some observations: (1) different inputs of segmenter statistics can be well combined; (2) some basic operators can effectively improve the correlation. Thus, we build computational graphs with multiple statistics as inputs and different advanced basis arithmetic as the primary operations to represent candidate proxies. Then, we employ an evolutionary algorithm to crossover and mutate the superior candidates in the population based on correlation evaluation. Finally, based on the searched proxy, we perform the segmenter search without candidate training. In this way, SasWOT not only enables automated proxy optimization for SAS tasks but also achieves significant search acceleration before the retrain stage. Extensive experiments on Cityscapes and CamVid datasets demonstrate that SasWOT achieves superior trade-off between accuracy and speed over several state-of-the-art techniques. More remarkably, on Cityscapes dataset, SasWOT achieves the performance of 71.3% mIoU with the speed of 162 FPS.

Semantic segmentation of progressive micro-cracking in polymer composites using Attention U-Net architecture

The present study delivers a methodology for investigating the gradual damage development in a carbon fibre-reinforced cross-ply polymer composite during a sequence of thermo-mechanical loadings with the help of X-ray computed tomography. The procedure allows an in-depth analysis of the occurrence and nature of the multiple cracks that form within layers oriented perpendicular, or transverse, to the loading direction. This is achieved by using Attention U-Net architecture for semantic segmentation of the transverse cracks. The model shows promising results, through an ability to identify all the transverse cracks and reflect the damage progression. The described method provides a robust routine for analysing challenging polymer composite tomographic datasets.

Combination of Image Enhancement and U-Net Architecture for Cervical Cell Semantic Segmentation

Cervical cancer is the second leading cause of death in women and ranks fourth as a disease that occurs in women worldwide. Cervical cancer is a disease that is difficult to detect and can be detected when it is in an advanced stage. This requires early prevention by carrying out a pap-smear examination. Pap-smear examination manually requires a relatively long time, so a tool is needed by segmentation. Segmentation is image processing by performing perfection between the intended object and the background. One of the CNN methods commonly used in medical image segmentation is the U-Net architecture. Segmentation in this study was carried out on the nucleus and cytoplasm of the Herlev dataset using the U-Net architecture combined with data augmentation and image enhancement. In the learning process, this research resulted in a fairly high IoU value of 78% and an RMSE close to 20%. The results of this study also yielded an accuracy value of 89%, with an average precision, recall and F1 score of 89%, 89% and 88.67%, respectively. This shows that the combination of the CNN U-Net architecture with image quality improvement and data augmentation is quite good at segmenting cervical cells for the nucleus and cytoplasm

Real-time reliable semantic segmentation of thyroid nodules in ultrasound images

Objective. Low efficiency in medical image segmentation is a common issue that limits computer-aided diagnosis development. Due to the varying positions and sizes of nodules, it is not easy to accurately segment ultrasound images. This study aims to propose a segmentation model that maintains high efficiency while improving accuracy. Approach. We propose a novel layer that integrates the advantages of dense connectivity, dilated convolution, and factorized filters to maintain excellent efficiency while improving accuracy. Dense connectivity optimizes feature reuse, dilated convolution redesigns layers, and factorized convolution improves efficiency. Moreover, we propose a loss function optimization method from a pixel perspective to increase the network's accuracy further. Main results. Experiments on the Thyroid dataset show that our method achieves 81.70% intersection-over-union (IoU), 90.50% true positive rate (TPR), and 0.25% false positive rate (FPR). In terms of accuracy, our method outperforms the state-of-the-art methods, with twice faster inference and nearly 400 times fewer parameters. Meanwhile, in a test on an External Thyroid dataset, our method achieves 77.03% IoU, 82.10% TPR, and 0.16% FPR, demonstrating our proposed model's robustness. Significance. We propose a real-time semantic segmentation architecture for thyroid nodule segmentation in ultrasound images called fully convolution dense dilated network (FCDDN). Our method runs fast with a few parameters and is suitable for medical devices requiring real-time segmentation.

Improving Three-Dimensional Building Segmentation on Three-Dimensional City Models through Simulated Data and Contextual Analysis for Building Extraction

Digital twins are increasingly gaining popularity as a method for simulating intricate natural and urban environments, with the precise segmentation of 3D objects playing an important role. This study focuses on developing a methodology for extracting buildings from textured 3D meshes, employing the PicassoNet-II semantic segmentation architecture. Additionally, we integrate Markov field-based contextual analysis for post-segmentation assessment and cluster analysis algorithms for building instantiation. Training a model to adapt to diverse datasets necessitates a substantial volume of annotated data, encompassing both real data from Quebec City, Canada, and simulated data from Evermotion and Unreal Engine. The experimental results indicate that incorporating simulated data improves segmentation accuracy, especially for under-represented features, and the DBSCAN algorithm proves effective in extracting isolated buildings. We further show that the model is highly sensible for the method of creating 3D meshes.

DRD-UNet, a UNet-Like Architecture for Multi-Class Breast Cancer Semantic Segmentation

DRD-UNet, a UNet-Like Architecture for Multi-Class Breast Cancer Semantic Segmentation

A modified UNet-based semantic segmentation architecture for pancreas tumour detection

A modified UNet-based semantic segmentation architecture for pancreas tumour detection

Attention-Based Semantic Segmentation Networks for Forest Applications

Deforestation remains one of the key concerning activities around the world due to commodity-driven extraction, agricultural land expansion, and urbanization. The effective and efficient monitoring of national forests using remote sensing technology is important for the early detection and mitigation of deforestation activities. Deep learning techniques have been vastly researched and applied to various remote sensing tasks, whereby fully convolutional neural networks have been commonly studied with various input band combinations for satellite imagery applications, but very little research has focused on deep networks with high-resolution representations, such as HRNet. In this study, an optimal semantic segmentation architecture based on high-resolution feature maps and an attention mechanism is proposed to label each pixel of the satellite imagery input for forest identification. The selected study areas are located in Malaysian rainforests, sampled from 2016, 2018, and 2020, downloaded using Google Earth Pro. Only a two-class problem is considered for this study, which is to classify each pixel either as forest or non-forest. HRNet is chosen as the baseline architecture, in which the hyperparameters are optimized before being embedded with an attention mechanism to help the model to focus on more critical features that are related to the forest. Several variants of the proposed methods are validated on 6120 sliced images, whereby the best performance reaches 85.58% for the mean intersection over union and 92.24% for accuracy. The benchmarking analysis also reveals that the attention-embedded high-resolution architecture outperforms U-Net, SegNet, and FC-DenseNet for both performance metrics. A qualitative analysis between the baseline and attention-based models also shows that fewer false classifications and cleaner prediction outputs can be observed in identifying the forest areas.

MSSFF: Advancing Hyperspectral Classification through Higher-Accuracy Multistage Spectral–Spatial Feature Fusion

This paper presents the MSSFF (multistage spectral–spatial feature fusion) framework, which introduces a novel approach for semantic segmentation from hyperspectral imagery (HSI). The framework aims to simplify the modeling of spectral relationships in HSI sequences and unify the architecture for semantic segmentation of HSIs. It incorporates a spectral–spatial feature fusion module and a multi-attention mechanism to efficiently extract hyperspectral features. The MSSFF framework reevaluates the potential impact of spectral and spatial features on segmentation models and leverages the spectral–spatial fusion module (SSFM) in the encoder component to effectively extract and enhance these features. Additionally, an efficient Transformer (ET) is introduced in the skip connection part of deep features to capture long-term dependent features and extract global spectral–spatial information from the entire feature map. This highlights the significant potential of Transformers in modeling spectral–spatial feature maps within the context of hyperspectral remote sensing. Moreover, a spatial attention mechanism is adopted in the shallow skip connection part to extract local features. The framework demonstrates promising capabilities in hyperspectral remote sensing applications. The conducted experiments provide valuable insights for optimizing the model depth and the order of feature fusion, thereby contributing to the advancement of hyperspectral semantic segmentation research.

A multi-objective comparison of CNN architectures in Arctic human-built infrastructure mapping from sub-meter resolution satellite imagery

ABSTRACTRisk assessment of infrastructure exposed to ice-rich permafrost hazards is essential for climate change adaptation in the Arctic. As this process requires up-to-date, comprehensive, high-resolution maps of human-built infrastructure, gaps in such geospatial information and knowledge of the applications required to produce it must be addressed. Therefore, this study highlights the ongoing development of a deep learning approach to efficiently map the Arctic built environment by detecting nine different types of structures (detached houses, row houses, multi-story blocks, non-residential buildings, roads, runways, gravel pads, pipelines, and storage tanks) from recently-acquired Maxar commercial satellite imagery (<1 m resolution). We conducted a multi-objective comparison, focusing on generalization performance and computational cost, of nine different semantic segmentation architectures. K-fold cross validation was used to estimate the average F1-score of each architecture and the Friedman Aligned Ranks test with the Bergmann-Hommel post-hoc procedure was applied to test for significant differences in generalization performance. ResNet-50-UNet++ performs significantly better than five out of the other eight candidate architectures; no significant difference was found in the pairwise comparisons of ResNet-50-UNet++ to ResNet-50-MANet, ResNet-101-MANet, and ResNet-101-UNet++. We then conducted a high-performance computing scaling experiment to compare the number of service units and runtime required for model inferencing on a hypothetical pan-Arctic scale dataset. We found that the ResNet-50-UNet++ model could save up to ~ 54% on service unit expenditure, or ~ 18% on runtime, when considering operational deployment of our mapping approach. Our results suggest that ResNet-50-UNet++ could be the most suitable architecture (out of the nine that were examined) for deep learning-enabled Arctic infrastructure mapping efforts. Overall, our findings regarding the differences between the examined CNN architectures and our methodological framework for multi-objective architecture comparison can provide a foundation that may propel future pan-Arctic GeoAI mapping efforts of infrastructure.

SEMANTIC SEGMENTATION OF REMOTE SENSING IMAGERY USING AN ENHANCED ENCODER-DECODER ARCHITECTURE

Abstract. Semantic segmentation is one of most the important computer vision tasks for the analysis of aerial imagery in many remote sensing applications, such as resource surveys, disaster detection, and urban planning. This area of research still faces unsolved challenges, especially in cluttered environments and complex sceneries. This study presents a repurposed Robust UNet (RUNet) architecture for semantic segmentation, and embeds the architecture with attention mechanism in order to enhance feature extraction and construction of segmentation maps. The attention mechanism is achieved using Squeeze-and-Excitation (SE) block. The resulting network is referred to as SE-RUNet. SE is also tested with the classical UNet, termed SE-UNet, to verify the efficiency of introducing SE. The proposed approach is trained and tested using “Semantic Segmentation of Aerial Imagery” dataset. The results are evaluated using Accuracy, Precision, Recall, F-score and mean Intersection over Union (mIoU) metrics. Comparative evaluation and experimental results show that using SE to embed attention mechanism into UNet and RUNet significantly improves the overall performance.

Enhanced multi-scale networks for semantic segmentation

Multi-scale representation provides an effective answer to the scale variation of objects and entities in semantic segmentation. The ability to capture long-range pixel dependency facilitates semantic segmentation. In addition, semantic segmentation necessitates the effective use of pixel-to-pixel similarity in the channel direction to enhance pixel areas. By reviewing the characteristics of earlier successful segmentation models, we discover a number of crucial elements that enhance segmentation model performance, including a robust encoder structure, multi-scale interactions, attention mechanisms, and a robust decoder structure. The attention mechanism of the asymmetric non-local neural network (ANNet) is merged with multi-scale pyramidal modules to accelerate model segmentation while maintaining high accuracy. However, ANNet does not account for the similarity between pixels in the feature map channel direction, making the segmentation accuracy unsatisfactory. As a result, we propose EMSNet, a straightforward convolutional network architecture for semantic segmentation that consists of Integration of enhanced regional module (IERM) and Multi-scale convolution module (MSCM). The IERM module generates weights using four or five-stage feature maps, then fuses the input features with the weights and uses more computation. The similarity of the channel direction feature graphs is also calculated using ANNet’s auxiliary loss function. The MSCM module can more accurately describe the interactions between various channels, capture the interdependencies between feature pixels, and capture the multi-scale context. Experiments prove that we perform well in tests using the benchmark dataset. On Cityscapes test data, we get 82.2% segmentation accuracy. The mIoU in the ADE20k and Pascal VOC datasets are, respectively, 45.58% and 85.46%.

A Semantics-Guided Visual Simultaneous Localization and Mapping with U-Net for Complex Dynamic Indoor Environments

Traditional simultaneous localization and mapping (SLAM) system tends to operate in small-area static environments, and its performance might degrade when moving objects appear in a highly dynamic environment. To address this issue, this paper proposes a dynamic object-aware visual SLAM algorithm specifically designed for dynamic indoor environments. The proposed method leverages a semantic segmentation architecture called U-Net, which is utilized in the tracking thread to detect potentially moving targets. The resulting output of semantic segmentation is tightly coupled with the geometric information extracted from the corresponding SLAM system, thus associating the feature points captured by images with the potentially moving targets. Finally, filtering out the moving feature points can greatly enhance localization accuracy in dynamic indoor environments. Quantitative and qualitative experiments were carried out on both the Technical University of Munich (TUM) public dataset and the real scenario dataset to verify the effectiveness and robustness of the proposed method. Results demonstrate that the semantics-guided approach significantly outperforms the ORB SLAM2 framework in dynamic indoor environments, which is crucial for improving the robustness and reliability of the SLAM system.

SwinCross: Cross-modal Swin transformer for head-and-neck tumor segmentation in PET/CT images.

Radiotherapy (RT) combined with cetuximab is the standard treatment for patients with inoperable head and neck cancers. Segmentation of head and neck (H&N) tumors is a prerequisite for radiotherapy planning but a time-consuming process. In recent years, deep convolutional neural networks (DCNN) have become the de facto standard for automated image segmentation. However, due to the expensive computational cost associated with enlarging the field of view in DCNNs, their ability to model long-range dependency is still limited, and this can result in sub-optimal segmentation performance for objects with background context spanning over long distances. On the other hand, Transformer models have demonstrated excellent capabilities in capturing such long-range information in several semantic segmentation tasks performed on medical images. Despite the impressive representation capacity of vision transformer models, current vision transformer-based segmentation models still suffer from inconsistent and incorrect dense predictions when fed with multi-modal input data. We suspect that the power of their self-attention mechanism may be limited in extracting the complementary information that exists in multi-modal data. To this end, we propose a novel segmentation model, debuted, Cross-modal Swin Transformer (SwinCross), with cross-modal attention (CMA) module to incorporate cross-modal feature extraction at multiple resolutions. We propose a novel architecture for cross-modal 3D semantic segmentation with two main components: (1) a cross-modal 3D Swin Transformer for integrating information from multiple modalities (PET and CT), and (2) a cross-modal shifted window attention block for learning complementary information from the modalities. To evaluate the efficacy of our approach, we conducted experiments and ablation studies on the HECKTOR 2021 challenge dataset. We compared our method against nnU-Net (the backbone of the top-5 methods in HECKTOR 2021) and other state-of-the-art transformer-based models, including UNETR and Swin UNETR. The experiments employed a five-fold cross-validation setup using PET and CT images. Empirical evidence demonstrates that our proposed method consistently outperforms the comparative techniques. This success can be attributed to the CMA module's capacity to enhance inter-modality feature representations between PET and CT during head-and-neck tumor segmentation. Notably, SwinCross consistently surpasses Swin UNETR across all five folds, showcasing its proficiency in learning multi-modal feature representations at varying resolutions through the cross-modal attention modules. We introduced a cross-modal Swin Transformer for automating the delineation of head and neck tumors in PET and CT images. Our model incorporates a cross-modality attention module, enabling the exchange of features between modalities at multiple resolutions. The experimental results establish the superiority of our method in capturing improved inter-modality correlations between PET and CT for head-and-neck tumor segmentation. Furthermore, the proposed methodology holds applicability to other semantic segmentation tasks involving different imaging modalities like SPECT/CT or PET/MRI. Code:https://github.com/yli192/SwinCross_CrossModalSwinTransformer_for_Medical_Image_Segmentation.